169: Development of a Scale for Measuring Evidence-searching Capability: a Modified Delphi Study

Background and aims: The ability to acquire the best evidence efficiently is important for busy healthcare professionals who have to make decision within a limited time. However, the current available assessment tools in evidence-based medicine (EBM), e.g. Berlin questionnaire and Fresno test, were not designed for examining evidence-searching capability. The objective of this study was to develop a scale for measuring evidence-searching skills. Methods: We use 2-round modified Delphi technique and organised a workforce of 7 experts who provided comments on a draft 33-item scale and rated each item on a 5-point Likert-type scale. All items rated less than 3 by any expert were removed. The items were modified or merged after authors’ discussion considering experts’ comments. When all items were rated ≥3 by all experts with an interquartile range (IQR) of ≤1, a consensus was achieved. In a pilot test, the searching capability of two examinees was assessed by two raters and re-assessed two weeks later for reliability. In a further pilot test, the searching capability of another eight examinees was assessed by two raters and the reliability of the scale was examined. Results: All 7 experts completed the two rounds. In Round 1, 33 items were removed and 11 items were merged into 4 items, leaving 17 items remained in the scale. In Round 2, 17 items were removed. The final consensus scale consisted of 15 items, all rated 4 or 5 with a mean of 4.79 and an IQR of 1 or less. The full details of the scale will be presented. In the pilot tests, the inter-rater correlation was 0.911 (95% confidence interval 0.821–0.956), while the intra-rater correlation coefficient was 1. The Cronbach’s alpha was 0.903. Conclusion: This study is the first to develop a scale for measuring evidence-searching skills. The scale fills in the gap in objective assessment of knowledge-acquiring ability, and may be used in improving the training of evidence-searching skills.